
STEREO IMAGE MEASURING DEVICE 

PACKGROT TND OF THE INVENTION 

The present invention relates to a stereo image measuring device 
and, more particularly, to a device for measuring a three-dimensional image 
from a stereo image. Throughout the specification, "off-line measurement" 
means executions of image photographing, for example, at a measuring site, 
and of operations other than the photographing at a different calculation 
and analyzing site. On the other hand, "on-line measurement" means 
executions of image measuring, and also of displaying of a three- 
dimensional image, for example, at a measuring site. 

In the conventional art of digital photogrammetry, three- 
dimensional measurement and analysis from an obtained digital image have 
been carried out in the following manner. First, orientation is carried out 
based on several imprinted control points to provide stereoscopic vision. 
Then, stereo matching is carried out from a plurality of obtained images to 
calculate a three-dimensional coordinate, and thus a three-dimensional 
image is analyzed. In the stereo matching in particular, a coarse-to-fine 
image correlation processing method or the like is used in order to surely 
obtain three-dimensional data. Here, according to the coarse-to-fine image 
correlation processing method, correlation processing is carried out in stages 
gradually from a low-resolution image to a high-resolution image, rather 
than executing correlation processing among high-resolution images from 
the beginning. This method reduces the number of local errors, thereby 
enhancing reliability (see Image Analysis Handbook, p. 709, compiled under 
the supervision of M. Takagi, and Y. Shimoda). The three-dimensional 
measurement has also been carried out by utilizing a survey instrument 




such as a total station, a GPS or the like. 

In addition, . to measure a relatively large object in the form of a 
three-dimensional image in the conventional art, the points of measurement 
have been scanned by use of the survey instrument such as a total station, 
or the points of measurement have been increased by moving the GPS, and 
thus the three-dimensional image has been displayed by a collection of a 
number of small surfaces surrounded with the plurality of points of 
measurement. Another method has also been available, that is, three- 
dimensional measurement has been carried out by executing image 
correlation processing (stereo matching) or the like from a photographed 
stereo image, and then displaying a three-dimensional image as a collection 
of a number of surface shapes. 

SUMMARY OF THE TNVENTION 

However, in such a conventional method, image correlation 
processing must be carried out even for each image having different 
resolution, thus necessitating a great deal of time for its arithmetic 
operation. In the conventional case, if a measuring error occurs, the 
operation of repeating the image correlation processing must be carried out 
by specifying the place of the measuring error again by an operator, thereby 
necessitating much labor and time for measurement and correction. Also, 
conventionally, when the tree dimensional measurement and analysis are 
needed at a natural disaster site or a place requiring emergency attendance 
or the like, there may be such cases where enough time may not be provided 
for measurement by the total station or the like at the actual site, 
measurement may be impossible, only photographs of the place may be 
taken, and the like. 




Especially with regard to the case of using the total station (TS), the 
conventional art has presented a method for obtaining three-dimensional 
data by automatically scanning the points of measurement by use of an 
auto-tracking total station (reflectorless TS). This reflectorless TS is a 
total station, which needs no reflectors such as prisms or the like. However, 
this method necessitates a great deal of time for measurement, because the 
measurement is carried out while driving the TS by a motor. For example, 
assuming that measurement of one point takes 1 sec, about 11 hours are 
necessary only for measuring 200x200 points. Other problems have 
included the impossibility of measurement caused by the unreturning of a 
ranging light, a larger reduction in accuracy as a ranging beam is farther, 
and so on. 

To move the GPS as in the case of the conventional art, much labor 
and time must be expended because of the measurement of a measurement 
point coordinate while moving a platform for loading the GPS. The need to 
move the GPS to a point to be measured has also made it impossible to 
perform measurement at a dangerous place. 

In the case of the GPS, problems regarding the measurement carried 
out by photographing from a stereo image has included the necessity of 
setting a control point (point of orientation) on an object to be measured, the 
impossibility of determining without orientation whether the photographed 
stereo image is measured or not, a longer calculation time for stereo 
matching (image correlation processing), the impossibility of measuring 
characterless places, and so on. In particular, to assure the execution of 
image correlation processing, a method called coarse -to-fine image 
correlation has been employed. According to this method, correlation 
processing is carried out in stages gradually from a low-resolution image to 



a high-resolution image, rather than executing correlation processing among 
high-resolution images from the beginning. Thus, the number of local 
errors is reduced, enhancing reliability (see Image Analysis Handbook, p709, 
compiled under the supervision of M. Takagi, and Y. Shimoda). In the case 
of this method, however, image correlation processing must be carried out 
even for each image having different resolution, necessitating a great deal of 
time for an arithmetic operation. 

Furthermore, in the case of measurement carried out by a stereo 
camera having a base line fixed, such an advantage is provided that a 
control point is unnecessary. However, there have been problems inherent, 
which include limitation on the angle of view (range of measurement), the 
necessity of a connection point (control point) for image connection caused 
by the increase of the range of measurement. 

The present invention was made with foregoing problems in mind, 
and it is an object of the invention to provide a measuring device capable of 
executing high-speed and highly reliable three-dimensional measurement 
for a stereo image, and easily correcting any omitted measurement, 
measurement errors or the like. It is another object of the invention to 
provide a measuring device capable of executing highly reliable and high- 
speed three-dimensional measurement and analysis only by measuring 
several control points and photographing a stereo image even in a site 
requiring emergency attendance because of natural disaster or the like, or 
when quick on site measurement is necessary or when measurement is 
difficult. 

It is another object of the invention to achieve high reliability and 
speeding up of analysis processing by enabling efficient execution of image 
correlation processing. It is also an object of the invention to carry out 



high-speed three-dimensional measurement in non-contact even in an area 
conventionally impossible to be measured, such as a place too dangerous to 
enter or the like. 

It is a further object of the invention to achieve, especially in the 
case of on-line measurement, much higher speed, efficiency and high 
reliability of entire measurements compared with the case where single 
measurement is carried out by each method. 

In order to achieve the foregoing object, according to the invention, 
especially in the case of off-line measurement, a stereo image measuring 
device is provided, comprising: a setting unit for setting, regarding a stereo 
image including at least three or more points of measurement having 
position data thereof obtained, at least a part of the points of measurement 
as division points, and then setting a search area based on at least three 
selected from a plurality of the set division points** an arithmetic operation 
unit for executing correlation processing for images of search areas 
corresponding to each other on the stereo image based on the search area 
set by the setting unit; and a measuring unit for measuring a coordinate of a 
point in a given position from a result of the correlation executed by the 
arithmetic operation unit. 

In order to achieve the foregoing object, according to the invention, 
especially in the case of on-line measurement, a stereo image measuring 
device is provided, comprising: a setting unit for setting, regarding a stereo 
image including at least three or more points of measurement having 
position data thereof obtained, at least a part of the points of measurement 
as division points, and then setting a search area based on at least three 
selected from a plurality of the set division points; an arithmetic operation 
unit for executing correlation processing for images of search areas 




corresponding to each other on the stereo image based on the search area 
set by the setting unit; and a measuring unit for measuring a coordinate of a 
point in a given position based on a result of the correlation executed by the 
arithmetic operation unit, where the arithmetic operation unit prepares 
information regarding an area of measurement which requires a new point 
of measurement, according to the result of the correlation processing. 

Mainly in the case of on-line measurement, the invention has, for 
example, the following features in order to achieve the foregoing object. By 
executing these processing below, it is possible to carry out measurement at 
a higher speed and with higher reliability than those in the conventional 
case. 

1. Photographing of a stereo image of an area to be measured. 

2. Measurement of six or more control (orientation) points in an area 
included in both stereo images by a reflectorless TS. 

3. Measurement (orientation) of control (orientation) points measured by TS 
on the stereo image. 

4. Decision of an area to be subjected to image correlation processing based 
on points of measurement. 

5. Execution of image correlation processing for an area. 

6. Displaying, on a display, of an image formed based on three-dimensional 
coordinates obtained from a correlation coefficient or the result of 
correlation processing of step 5. 

Further, mainly in the case of on line measurement, the invention 
has the following features. The addition of these processings enables 
measurement to be executed at a much higher speed and with much higher 
reliability. If the reliability of measurement is considered to be insufficient, 
the following operations 7 to 10 may be repeated until satisfactory reliability 




is provided, 

7. Execution of additional measurement for an area having a low correlation 
coefficient, or a place where an image formed based on measurement data is 
not satisfactory. 

8. Re-decision of an area to be subjected to image correlation processing 
based on a point of measurement. 

9. Execution of image correlation processing for an area. 

10. Displaying, on a display, of an image formed from the result of 
correlation. 

Furthermore, according to the invention, mainly in the on-line 
measurement, for an area that cannot be measured by the reflectorless TS, 
and not measured well even by image correlation processing, calculation can 
be executed by interpolating an elevation value from information on the 
measurement area of the reflectorless TS. The advantages of such 
processing are as follows : 

• Since the three-dimensional measurement value of the reflectorless TS can 
be set as an initial value for stereo matching, coarse -to -fine image 
correlation processing can be omitted, making it possible to shorten the time 
of calculation. 

• The combination measurement using a stereo image and the reflectorless 
TS enables the total time of measurement to be much faster than that in 
single measurement using one of TS and a stereo image. 

• A place where image measurement is difficult (e.g., characterless) can be 
measured by the reflectorless TS, or conversely, a place where no light 
returns for the reflectorless TS can be measured by an image. Thus, both 
measuring difficult places can be interpolated. 

• For an area of bad image correlation, interpolation is performed by the 




measurement of the reflectorless TS and, further, by setting its 
measurement value as an initial value, the area of measurement can be 
finer and more proper, thus enhancing reliability. 

• Since on-site measurement is enabled, and measurement can be carried 
out while verifying an error area, the reliability of measurement can be 
enhanced, eliminating failures. 

BRIEF DESCRTPTTON OF THE DRAWINGS 

Fig. 1 is a block diagram of an entire measuring device according to 
the present invention. 

Fig. 2 is a flowchart of on-site work. 

Fig. 3 is a view illustrating a stereo image. 

Fig. 4 is a flowchart of pre-processing. 

Fig. 5 is a flowchart of off-line measurement. 

Fig. 6 is a view illustrating formation of a triangle based on a point 
of measurement. 

Fig. 7 is a view illustrating an input image and a template image. 
Fig. 8 is a view showing an example of 3x3 pixels. 
Fig. 9 is a view illustrating a stereo method. 

Figs. 10(a) and 10(b) are views, each thereof illustrating displaying 
of a result of correlation. 

Fig. 11 is a view illustrating an additional point of measurement. 

Figs. 12(a) and 12(b) are views, each thereof illustrating a triangle 
recombination. 

Fig. 13 is a flowchart of automatic measurement of three- 
dimensional coordinates for a representative point. 

Fig. 14 is a view illustrating representative points to be 



automatically measured. 

Fig. 15 is a flowchart of on-line automatic measurement. 
Fig. 16 is a flowchart of graphic displaying. 

Figs. 17(a) and 17(b) are views, each thereof illustrating graphic 
displaying. 

Fig. 18 is a view (l) illustrating setting of a search area. 

Fig. 19 is a view illustrating a search area. 

Fig. 20 is a view (2) illustrating setting of a search area. 

Figs. 21(a) and 21(b) are views, each thereof illustrating specifying 
of a size of a search area. 

Figs. 22(a) and 22(b) are views (3), each thereof illustrating setting 
of a search area. 

Figs. 23(a) and 23(b) are views, each thereof illustrating a center of 
gravity of a triangle and a data block. 

Figs. 24(a) and 24(b) are views, each thereof illustrating a size of a 
data block. 

Fig. 25 is a flowchart of on-line measurement. 

Fig. 26 is a flowchart of semi automatic measurement. 

DETAILED DESCRTPTTON OF THE PREFERRED EMBODIMENTS 

Next, description will be made of the preferred embodiments of the 
present invention with reference to the accompanying drawings. 

A: Hardware 

Fig. 1 is a block diagram showing the entire measuring device of the 
invention. 

This system comprises a survey instrument 1, a measuring device 2, 
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and a camera 3. The measuring device 2 includes a control point searching 
unit 10, a search area setting unit 20, an arithmetic operation unit 30, a 
display unit 40, a measuring unit 50, and a bus 60. These components, i.e., 
the control point searching unit 10, the search area setting unit 20, the 
arithmetic operation unit 30, the display unit 40 and the measuring unit 50 
are loaded on, for example a personal computer, and interconnected through 
the bus 60. 

The survey instrument 1 is used to measure several control points 
on site. This instrument is not necessary when measurement is performed 
by obtaining a stereo image having several control points contained therein 
beforehand. The camera 3 is used to obtain an image, and one can be 
selected for use from, for example a digital camera, a film camera, and so on. 
It is not always necessary to prepare the camera 3. Instead, an image 
having several control points imprinted therein beforehand may be obtained 
and analyzed. 

The control point searching unit 10 correlates a control point 
measured beforehand by the survey instrument 1, with an image. The 
search area setting unit 20 sets, based on the control point correlated by the 
control point searching unit 10, a search area for image correlation 
processing, and reference data and search data blocks. The arithmetic 
operation unit 30 executes orientation calculation and image correlation 
processing (stereo matching) for the search area set by the search area 
setting unit 20. The display unit 40 is a stereo monitor for providing 
stereoscopic vision, a personal computer monitor or the like. The use of 
such a stereo monitor enables more specific and accurate three-dimensional 
measurement to be performed, and the measurement result to be verified. 
The display unit 40 displays, in a graphic manner, a photographed stereo 
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image, a correlation coefficient value obtained from the result of correlation 
processing, a point and a contour line formed based on three-dimensional 
coordinates obtained from the result of correlation processing, a bird's-eye 
view, a digital ortho-image, and so on. The method using the display unit 
40 to execute graphic displaying, verification or additional measurement can 
be used for, for example, proper visual determination of the result of 
measurement, an object having not so high correlation coefficient reliability 
and the like. The graphic displaying can be executed in real time. 

The measuring unit 50 executes additional measurement when the 
result of correlation processing is undesirable. For the additional 
measurement by the measuring unit 50, such a method may be used, where 
correlation coefficients are displayed and then additional measurement is 
executed for a portion having little correlation. Another possible method 
may be used, which displays in a graphic manner a point and a contour line 
based on three-dimensional coordinates of each point, a wire frame model, a 
surface model having a stuck surface, a bird's eye view or a digital ortho- 
image made to have a stuck image, or the like, and then executes additional 
measurement for the defective point of the displayed image. Regarding the 
method of displaying correlation coefficients on a screen to execute image 
verification or correction, one may be selected for use from the following 
methods: displaying the area of a low correlation coefficient and executing 
manual or semi-automatic measurement correction based on the screen; 
using correlation coefficients, and executing automatic measurement for an 
area of a specified threshold or lower; and so on. 

B: Off-line measurement 

Next, description will be made of "off-line measurement" included in 
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the operation of the stereo image measuring device of the invention. 

Then ^ manual or semi-automatic measurement, and automatic 
measurement will be described in detail. 

B-1. On-site work 

First, on-site work is explained. Fig. 2 is a flowchart of on-site 
work. If measurement is started on a site, the steps of this flowchart are 
carried out as on-site work. If a stereo image having a control point 
contained therein has already been obtained, these steps are not necessary. 

In step SIO, first, the stereo image of an area to be measured on the 
site is photographed. Fig. 3 illustrates a stereo image. As shown, two 
overlapped stereo images (left image L, and right image R) are 
photographed. Then, in step S20, at the survey instrument 1, three or 
more control (orientation) points are photographed in an area included in 
both of the stereo-photographed left and right images. The number of 
control points may be 3 or more. However, if the number of control points 
is 6 or more, orientation processing (see later step S40) will be more stable, 
enhancing the reliability of analysis carried out later. Thus, here, 6 points 
will be taken as an example. Specifically, in the area having the left and 
right images overlapped in Fig. 3, 6 points, i.e., CI to C6 are set to be control 
(orientation) points, and measured by the survey instrument 1. Then, 
stereo image data and a control point coordinate value measured by the 
survey instrument are transferred to a place, e.g., an office, where the stereo 
image measuring device is installed. These data can be transmitted by use 
of a memory medium for storing an image or through a telephone line or the 
like. 




B-2. Pre-processing 

Fig. 4 is a flowchart of pre-processing. After the entry of the image 
data and the control point coordinate value into the stereo image measuring 
device, the pre-processing like shown in the drawing is executed. 

First, in step S30, the transferred stereo image is displayed on the 
display of the display unit 40, and this image is then verified by an operator. 
In step S40, on the left and right images displayed at the display unit 40, 
the control (orientation) points measured by the survey instrument 1 are 
subjected to measurement (correlation). In other words, in this example, 
the points Cl to C6 that have been measured by the survey instrument 1 
are then subjected to orientation respectively on the left and right images L 
and R. In step S50, orientation processing is executed based on the control 
points measured in step S40, and a rectified image is formed. In other 
words, an image to be stereoscopically viewed (measured) having vertical 
parallax eliminated is formed. In step S60, the stereo image to be 
stereoscopically viewed (measured), formed in step S50, is then displayed on 
the display of the display unit 40. 

In step S70, the operator executes measurement (formation of a 
graph) for a place particularly intended to be measured (formed into a 
graph), a boundary line, a place having a level difference or the like by use 
of a pointing device, e.g., a mouse, while watching the display. This 
measurement work (graph formation) is carried out based on a stereoscopic 
vision if a stereo monitor is available. Alternatively, the left and right 
images may be respectively displayed on the monitor of the personal 
computer and the points of correspondence in both left and right images 
may then be pointed (and measured). Otherwise, by means of semi- 
automatic correlation processing or the like, a place indicated on the left 
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image may be subjected to automatic correlation processing on the right 
image, and thereby a corresponding point may be searched and indicated. 
The use of such processing may greatly facilitate measurement (graph 
formation) work. This measurement work (graph formation) does not 
5 always have to be carried out in the above step. However, such execution 
enables measurement to be carried out with higher reliability, and 
correction work to be reduced accordingly. 



B-3. Basic measurement flow 

P 

^3 10 Fiff. 5 is a flowchart of off-line measurement. The process from step 

!:8 

S80 to SllO is automatically carried out by computer processing irrespective 
1 U of manual or semi-automatic measurement. 

m 

^ J In step S80, the points of measurement measured in step S40 or S70 

p are interconnected to form a triangle. Fig. 6 illustrates the formation of 

I L| 15 such a triangle based on the points of measurement. This example shows a 

13 case where the points selected from the points of measurement CI to C6 are 

interconnected to form a triangle. In this case, the points may also be 
interconnected to form not a triangle but a square. However, since a 
triangular shape enables the inside of an area to be divided much more 
20 finely than a square (two planes can be formed by 4 points in the case of the 
triangle, while only one plane is formed in the case of the square), accuracy 
and reliability can be enhanced. As a method for interpolating a three- 
dimensional coordinate by such random points, triangulated irregular 
network (TIN) can be used. The TIN is designed to form a mesh composed 
25 of a triangle. For details on the TIN, see "Computational Geometry and 
Geographical Information Processing", pl27, by M, Iri, and T, Koshizuka, 
"Voronoi View, Introduction to one basic geometrical data structure" ACM 



15 



Computing Surveys, Vol. 23, pp345 to 405, by Franz Aurenhammer, 
translated by A, Sugihara, or the like. 

In step S91, the control point searching unit 10 detects data 
regarding a reference data block (template) from one of the images, e.g., the 
left image. For example, this data is regarded as a triangle formed by 3 
division points, and a square including the triangle, or the like. In step S92, 
the position or the size of the template is decided based on a distance from 
each of the division points of the template or the like. In step S93, from the 
template of the left image measured on the image, a search area for 
executing image correlation processing is decided by the search area setting 
unit 20. In step S94, the search area decided for the other image, e.g., the 
right image is subjected to scanning by use of the template of the left image. 
Setting of the template and scanning of the search area will be described 
later. 

Subsequently, in step SlOO, for each divided area, image correlation 
processing between the template and the search area is executed at the 
arithmetic operation unit 30. Specifically, the corresponding point of the 
right image which corresponds to the left image is obtained (or the 
corresponding point of the left image which corresponds to the right image) 
by obtaining one having large correlation, and thereby a three-dimensional 
coordinate thereof is calculated. Regarding the image correlation 
processing (stereo matching), a residual sequential testing method (SSDA 
method), a mutual correlation coefficient method or the like can be used. 

Hereinbelow, the mutual correlation coefficient method suited to 
automatic measurement is explained. 
(Method based on mutual correlation coefficient) 

Fig. 7 illustrates an input image and a template image. As shown. 
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it is assumed that searching has been carried out for the template image by 
moving the template image of N^xNi pixels within a searching range (Mf 
Ni+l)^ in the input image of MixMj pixels, which is larger than the template 
image, and obtaining the left upper position of the template image so as to 
have a largest correlation coefficient r represented by an expression below. 

Fig. 8 shows an example of 3x3 pixels. In this example, a 
comparison image 1 having a correlation value equal to or larger than that 
of the template image T of the left image is searched on the same line 
(epipolar line) from the right image. In other words, the foregoing 
expression is calculated in a corresponding relation to each image. Then, 
by shifting such images one by one or by a specified number, one having a 
high correlation value is obtained. 



r = 



COVARIANCE OF TEMPLATE IMAGE 
AND COMPARISON IMAGE 



V JDISPERSION OJ^ ^ . (1) 



TEMPLATE IMAGE COMPARISON IMAGE / 



^r,=jjZ(T-T)iI,-I) (2) 

^T=^t(T-Ty- — (3) 

(4) 



DENSITY LEVEL OF COMPARISON IMAGE 
T,: DENSITY LEVEL OF TEMPLATE IMAGE 

tJ AVERAGE value 



According to the invention, as an example, the reference data block 
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is set as a template image from the left image, searching is executed by use 
of a search data block in the search area of the right image as a unit, and 
then a search data block as a portion of the right image matching the 
reference data block (template) of the left image is searched. The matching 
portion is a point having a largest correlation coefficient (approaching l). 

After the corresponding points of the left and right images have been 
obtained by the foregoing correlation processing, a three-dimensional 
coordinate value of each point of measurement is calculated based on the 
principle of a stereo method. Fig. 9 illustrates such a stereo method. 
(Stereo method) 

For simplicity, it is assumed that two cameras are used, the optical 
axes thereof are parallel to each other, distances "a" from the principal 
points of camera lenses to CCD surfaces are equal to each other, and each 
CCD is placed perpendicularly to the optical axis. 

A distance (length of base line) between the two optical axes is set as 

L. 

The following relation is established between the coordinates of 
points Pi (xi,yi) and Pg (xg, on an object^ 
Xi=ax/z (D 
yi=y2=ay/z (D 

X2-Xi=aL/z (3) (5) 

Here, the origin of the entire coordinate system (x, y, z) is set as the lens 
principal point of the camera 1. If an image photographed by the camera 1 
is set as a left image, and one by the camera 2 is set as a right image, the 
position of Xg (point having highest similarity) with respect to the position of 
the left image x^ is obtained by the image correlation processing. 
Accordingly, z is obtained from the expression @ and, by use of this value, x 
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and y are obtained from the expressions ® and (D. 

The measurement is explained by referring back to the flowchart. 
In step SllO, the obtained result of correlation between the left and right 
images is displayed on the display unit 40. In this case, a largest 
correlation coefficient (point of highest similarity) found as a result of 
searching the search data block is set as a similarity in the data block, and 
the result of the correlation is displayed on the image. 

Figs. 10(a) and Fig. 10(b) are views, each thereof illustrating the 
displaying of the result of correlation. For example, as shown in the El 
area of Fig. 10(a) and in Fig. 10(b), if there is an area or a point having a low 
correlation coefficient, such an area or point is highlighted. For the 
displaying method, actual correlation coefficient values may be displayed by 
use of different colors or the like, or displayed by providing plural stages of 
thresholds. In step S120, the process is finished if the result of displaying 
is satisfactory. 

Next, an additional measurement flow will be described. In step 
SI 30, if the result of displaying is not satisfactory, the process proceeds to 
step Si 50. In step Si 50, for the area of the unsatisfactory correlation 
result, the unsatisfactory portion is additionally measured in a manual or a 
semi-automatic manner, or points of measurement are selected from the 
result of correlation. Here, since the state is off-line, the data that has 
already been measured by the survey instrument 1 is used in the additional 
measurement. For example, for a point in the area El of a low correlation 
coefficient shown in Fig. 10(a), specification is made manually or semi* 
automatically by use of the pointing device such as a mouse or the like while 
watching the stereo monitor or the monitor of the personal computer, and 
additional measurement is executed for the specified area. In the 
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additional measurement, a graph may be formed (measurement may be 
made) at points, or with hnes such as boundary lines depending on an object. 
According to another method, regarding points in the area El of a low 
correlation coefficient shown in Fig. 10(a), if there is a relatively correct 
point of measurement having a high correlation coefficient, this point may 
be selected on the display and set as an additional point of measurement. 
In addition, if there is a point not used as a control point, among the points 
measured by the survey instrument, then such a point may be selected as a 
point of measurement. For example, Fig. 11 illustrates such an additional 
point of measurement. In the drawing, C7 denotes an added point of 
measurement. As a result, areas ElO, Ell and E12 are added. 
Alternatively, both of the foregoing two methods may be employed. Further, 
even for an area having not so bad a correlation coefficient, if the result of 
correlation verified by the display unit 40 is not correct, the foregoing work 
may be carried out for this area. Such work assures more accurate 
measurement. 

Then, the process proceeds to step S80. In step S80, the points of 
measurement that have been additionally measured are added to form a 
triangle. Fig. 11 shows the addition of such a point of measurement. 
Specifically, the reconnection of the points of measurement is carried out by 
use of the additional point C7 shown in Fig. 11. In this case, a more 
detailed triangle may be formed for the additional point as shown, or 
triangles may be formed all over again irrespective of the previous 
connection points. 

In step Si 50, if no additional measurement is carried out or no 
points to be selected are present, the triangles are recombined. Figs. 12(a) 
and 12(b) are views, each thereof illustrating the recombination of triangles. 



20 



For example, if there are triangles formed like those shown in Fig. 12(a), 
then the triangles can be recombined as in the case of Fig. 12(b). In 
addition, if the points linearly measured in step Si 50 are additionally 
selected, then a finer triangular shape can be formed with those points, 
enhancing both reliability and a density. 

Subsequently, as in the case of the foregoing, processing of each step 
after step S91 is carried out, and then in step SlOO, image correlation 
processing is executed for a new area. In this case, if a detailed triangle is 
formed, only a newly added triangle area is subjected to image correlation 
processing. In step SllO, the result of correlation is displayed on the 
display unit. In step Si 20, if the result is satisfactory, the process is 
finished. In step SI 30, if the displayed result is not satisfactory, the 
process proceeds to the additional measurement again. In step SI 50, 
additional measurement is carried out for another area intended to be 
additionally measured while watching the monitor. The process thereafter 
is similar to that in above-mentioned step S80 and after, and repeated until 
a satisfactory result is obtained. 

Assuming that no improvement is made even by any additional 
measurements, or no additional measurement is carried out, then the 
process proceeds to step S140. In step S140, for example if results are 
always unsatisfactory taken by any means for the area ElO of Fig. 11, an 
elevation value is interpolated based on the plane equation of the plane ElO 
composed of the points of measurement CI, C2 and C7. As an example, to 
obtain C8 in the area ElO, if the coordinates of C8 are (xg, ys, Zg^^ then the 
elevation value Zg of C8 is calculated in the following manner^ 

- (ax8+by8+d)/c (6) 
(Coefficients a, b, c and d are calculated from the tree-dimensional 
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coordinates of CI, C2 and C7). 

B-4. Automatic measurement 

Next, description will be made of automatic measurement. Fig. 13 
is a flowchart of automatic measurement carried out for a representative 
point on a three-dimensional coordinate. Fig. 14 is a view illustrating a 
representative point to be automatically measured. To perform on-site 
measurement, the process similar to that of Fig. 2 is carried out, and the 
pre-processing thereof is similar to that of the flow shown in Fig. 4. In Fig. 
4, step S70 may be either executed or unexecuted. The automatic 
measurement is started after the pre-processing or step S150, alternatively 
during the processing of step Si 50. 

In step S200, the overlapped portion of the photographed left and 
right images is a search area, the overlapped portion is shown in Fig. 14, 
and this area is divided into a plurality. Then, a point (search data block) 
near the center of each divided area is selected, and image correlation 
processing is executed so as to obtain the corresponding point of the left and 
right images to the selected point. This image correlation processing is 
carried out more surely by utilizing coarse-to-fine image correlation 
processing. Since searching is necessary only around the divided data 
block of the image, high speed processing can be realized. A part having a 
low correlation coefficient found as a result of the correlation processing is 
eliminated, and a corresponding point having a high correlation coefficient 
is obtained (e.g., a correlation coefficient of 0.8 or higher can be set). 
Assuming that if no proper corresponding point is present among the points 
initially searched (data blocks), then the position of the point (search data 
block) is changed to the periphery thereof, and searching is executed. In 



22 



this way, the three-dimensional coordinates of one representative point are 
obtained for each area. In the drawing, division is made into 4x4 areas. 
However, even if any way is employed for division, or the point of correlation 
processing is not set as a center, the point may be proper as long as it is 
present in the area. 

In step S210, the result of measurement of step S200 is displayed on 
the display of the display unit 40. If there are no problems, then the 
process proceeds to a next flow. If there are any errors, shortages and the 
like, correction is made in this step. The correction is carried out by 
changing the position of the data block in the correction area. This work 
can be carried out automatically, manually, or semi-automatically. 
Alternatively, the process may proceed to the next step without any 
verification. However, if the verification is executed, later correction work 
will be facilitated. 

Now, Fig. 15 is a flowchart of automatic measurement as a sequel of 
the foregoing. Steps herein are also automatically executed by a computer. 
After step S210 shown in Fig. 13, these steps are executed. In step S220, 
the points having been automatically measured for each area are connected 
to form a triangle. The connection may also be made by adding the points 
measured (formed into a graph) in step S40 or in step S70. Each 
processing of steps S90 to S260 and S280 is similar to that of each of steps 
S90 to SllO and S140. If the result of displaying is satisfactory, then the 
process is finished. 

In step S270, based on the result of correlation displayed in the 
foregoing step, determination is made as to whether the additional 
automatic measurement is executed or not. If automatic correction 
measurement is carried out, the process proceeds to step S300. In step 




S300, after the execution of image correlation processing of step S240, a 
threshold is set among correlation numerical values, and a point including a 
correlation coefficient equal to the threshold or lower is eliminated from the 
target of additional selection. For the threshold, it may be fixed at 0.5 or 
lower, for example, from the beginning, or it may be decided depending on 
an object. Thus, a point having a relatively high correlation coefficient is 
selected or added for automatic measurement. The point selected for 
additional measurement can be set in the gravity center position of the 
triangle of the triangular area having a low correlation coefficient, or in the 
center (center of gravity) of the area having a low correlation coefficient. 
Then, coarse-to-fine image correlation processing is executed for a point of a 
high correlation coefficient near the above point or for its periphery. For 
further assurance, the processing may be carried out for both of them, and if 
both are found to be identical corresponding points, then such a point is 
selected as an additional point of measurement. If measurement is carried 
out by the survey instrument or the like, and if any unused point is present 
in the area, then this point may be automatically selected. 

Next, description will be made of a method for obtaining the gravity 
center position of a triangle and the center of gravity of an area which has a 
low correlation coefficient. 

First, with regard to the center of gravity of a triangle, for example if 
a correlation coefficient in the triangle El (CI, C2 and C5) of Fig. 10(a) is 
low, then the center of gravity C7 (see Fig. 11) of the triangle is obtained as 
a next point of measurement in the following manner. That is, assuming 
that the coordinate values of CI, C2 and C5 are respectively Cl (Xi, Yi, Z^), 
C2 (Xg, Yg, Z2), and C5 (X5, Y5, Z5), X and Y coordinates for a next selection 
point of measurement C7 are obtained by the following expressions^ 
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X=(Xi4-X2+X^/3 
Y=(Yi+Y2+Y5)/3 



(7) 



(8) 



Alternatively, a center of gravity may be obtained from the 
distribution of areas which have low correlation coefficients shown in Fig. 
10(b) (e.g., by the moment method). In the moment method, and yg 
obtained by the following expressions are used as an X and Y coordinate for 
selection points of additional measurement. The number of selection points 
of additional measurement may be one or plural. 



(xg, yg): coordinates of gravity center position, cor(x,y): correlation value on 
(x,y) coordinates 

Subsequently, the process moves to step S220. After step S220, the 
process is similar to that of the foregoing additional measurement flow, and 
the similar process is repeated after step S220 until a satisfactory result is 
obtained. If no improvement is made no matter how many times additional 
selection measurement is carried out, then the process proceeds to step S280. 
In step S280, for example, if any satisfactory result is not obtained for the 
area of ElO shown in Fig. 11 by any means, an elevation value is 
interpolated based on the plane equation of the plane ElO composed of the 
points of measurement Cl, C2 and C7. 

C- On-line measurement 

Next, description will be made of "on-line measurement" included in 
the stereo image measuring device of the invention. 

The invention is advantageous in that on line measurement enables 
displaying, verification and correction to be easily performed on the site of 



Xg={Zx*{l-cor(x,y)}}/Z{l-cor(x,y)} 
yg={2:y*{l-cor(x,y)}}/S{l-cor(x,y)} 



(9) 



(10) 
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measurement in real time, even without using the stereo monitor or the Hke. 
Thus, it is possible to shorten the total time of measurement, and perform 
sure and highly reliable three-dimensional measurement. 

Now, detailed explanation will be made of the case of performing 
manual or semi-automatic measurement and the case of performing 
automatic measurement, based on the result of correlation. 

C-1. Basic measurement flow 

Fig. 25 is a flowchart of on-line measurement. The on-line 
measurement means, for example, measuring of an image on the site of 
measurement, and displaying of a three-dimensional image thereof. 
Among steps, steps similar to those of the foregoing off-line measurement 
will be omitted to an extent that description will not be redundant. 

First, steps SIO and S20 are similar to those of the off line 
measurement (see Fig. 2 and its description). Specifically, in step SIO, first 
a stereo image of an area to be measured is photographed on site. Then, in 
step S20, by the survey instrument 1, three or more control (orientation) 
points are measured in an area included in both left and right stereo- 
photographed images. 

Then, in step S40, on the left and right images displayed on the 
display unit 40, the control (orientation) points measured by the survey 
instrument 1 are measured (correlated). In other words, in this example, 
the points CI to C6 that have been measured by the survey instrument 1 
are respectively subjected to orientation on the left and right images L and 
R. 

In step S80, the points of measurement measured in step S40 are 
interconnected to form a triangle. The specific operation is similar to that 
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of the foregoing off-line measurement (see Fig. 2 and its description). 

The operations of steps S91 to S94, SlOO, SllO, S120 and S140 are 
also similar to those of the foregoing off-line measurement (see Fig. 5, and 
its description). In step S120, if a displayed result is satisfactory, then the 
process is finished. 

Now, additional measurement flow will be described. In step S130, 
if the displayed result is not satisfactory, then the process proceeds to step 
S150. In step S150, additional measurement is carried out by the survey 
instrument 1 for an area required for the measurement, e.g., an area having 
a low correlation coefficient. Here, if the measurement is to be made 
manually, the process moves to step S80. 

C-2. Semi-automatic measurement 

The semi-automatic measurement is carried out by using an auto- 
tracking total station. Fig. 26 is a flowchart of semi-automatic 
measurement. 

In step S510, for example as shown in Fig. 10(a), the El area having 
a low correlation coefficient is indicated by the pointing device (by a cursor 
in the case of a notebook computer, or a pen in the case of a pen computer). 
In step S520, position information regarding the x and y coordinates of the 
indicated point is transferred to the survey instrument 1 from the personal 
computer, and measurement is executed by driving the survey instrument 1 
by a motor so as to measure the position. A measurement command is 
automatically sent fi:om the personal computer to the survey instrument 1. 
Alternatively, a human operator may verify the position of the survey 
instrument and then issue a measurement command. If a place to be 
measured is an immeasurable point, another place measurable is searched 
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in the area and measured. In the semi-automatic measuring work using 
such images, compared with the manual measurement, the process of 
measurement executed while tracking with the survey instrument can be 
omitted. Thus, considerably high work efficiency can be achieved. In 
other words, since an image tracked by the survey instrument reflects only 
the local area of an object, it will be very difficult and take time to track a 
place to be additionally measured. On the other hand, the operation 
according to the invention is simple, and can be carried out easily by 
anybody, because it needs only clicking on the image. 

Subsequently, the process proceeds to step S80 of Fig. 25. The 
process of steps S80 and thereafter is similar to that of the foregoing (see 
Fig. 5 of the off- line measurement, and its description, or Fig. 25 of on-line 
measurement and its description). 

C-3. Automatic measurement \ 

Automatic measurement can be carried out by utilizing an auto- 
tracking total station for the survey instrument. 

Now, the automatic measurement will be described. 
In step SI 50, measurement can be automated by executing the 
additional measurement in the following manner. After the execution of 
image correlation processing in SI 00, a threshold is set among correlation 
coefficients, and additional measurement is carried out for an area including 
a correlation coefficient equal to the threshold or lower. The threshold may 
be fixed, for example, at 0.5 from the beginning, or may be decided 
depending on an object. The area of measurement is a triangular area 
including a correlation coefficient equal to the threshold or lower. For 
example, in Fig. 10(a), the area of measurement is the area El. The point 
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to be additionally measured by the survey instrument may be set in the 
gravity center position of a triangle in the triangular area of a low 
correlation coefficient. Alternatively, it may be a point which has a lowest 
correlation coefficient in the area. Otherwise, it may be set in the center 
(center of gravity) of the area of low correlation coefficient. A method 
employed to obtain the gravity center position of the triangle and the center 
of gravity of the area of a low correlation coefficient is similar to that of the 
foregoing off-line measurement, and thus description thereof will be omitted. 
Now, after the point of additional measurement has been decided, 
the survey instrument is controlled with respect to its coordinate value, and 
driven to execute measurement. If such a point of additional measurement 
proves to be immeasurable or difficult to be measured, then searching is 
carried out in the vicinity thereof, and measurement is executed. Other 
steps are executed by a computer, and thus a series of operations from step 
S80 to S150 are automated. 

D. Correction by graphic displaying 

Fig. 16 is a flowchart of graphic displaying. 

Now, description will be made of a method for verification and 
correction based on the result of correlation processing while executing 
graphic displaying. In other words, verification and correction can be 
realized by executing step SllO of Fig. 5 (or step S250 of Fig. 15) in the 
following manner. 

First, in step S102, the three-dimensional coordinates of the 
obtained points are interconnected to form a triangle. In step Si 06, a 
graphic display image is formed based on each of the three-dimensional 
coordinates. When necessary, an image including the respective points of 
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the three-dimensional coordinates and Unes interconnecting such points, 
contour lines and a bird's-eye view having a surface or an image stuck on 
the triangle is formed. A view including an image stuck on the triangle 
may be an orthogonal projected image. If not necessary (visually 
determinable), then such processing may be omitted (e.g., one formed by 
connecting triangles may only be used). In fact, any kind is used as long as 
three-dimensional coordinates can be visually shown. Such an image is 
displayed graphically to be properly determined on the stereo monitor, the 
personal computer monitor or the like. In step S106, the formed image is 
displayed on the display unit 40. For displaying, any methods may be used, 
which facilitate determination of image, depending on a situation, for 
example, triangles in an overlapped manner, correlation coefficients in an 
overlapped manner, or the like. 

For the bird*s-eye view image or the like formed/displayed in steps 
S102 to S106, additional measurement can be carried out for an error or 
unnatural area. Each of Figs. 17(a) and 17(b) illustrates graphic displaying. 
For example, even with merely the displaying of one formed by connecting 
triangles, if the image is different from an actual situation, a problem place 
can be determined from the displaying. For example, if an obtained three- 
dimensional coordinates are moved by mistake to Fl' of Fig. 17(b) while its 
original point is Fl of Fig. 17(a), such an error can be clearly verified. 
Further, by displaying a surface or image stuck to Fig. 17(b), a difference 
from an actual situation can be more clearly emphasized. In addition, by 
displaying overlapping with correlation coefficient values, if the area of a 
low correlation coefficient coincides with a problem area, the area to be 
corrected can be made much clearer. In this case, it is only necessary to 
carry out manual or semi-automatic additional measurement for an area 



30 



around the Fl of Fig. 17(b). 

In the case of manual on-Une measurement, the process moves to 
step S80. On the other hand, if the auto-tracking total station is used, 
semi-automatic measurement is enabled, and the process proceeds to step 
SI 10. In the semi automatic measurement, as described above, for 
example as shown in Figs. 17(a) and 17(b), processing thereafter is executed 
by indicating the error or unnatural area Fl with the cursor in the case of 
the notebook computer or the pen in the case of the pen computer. 

E. Method for setting a search area 

Next, description will be made of the setting and scanning of a 
search area, a reference data block and a search data block by the search 
area setting unit 20. The setting method of a search area is exempUfied in 
the following manner, and will be explained in order. 

1. Set a search area^ inclusion square 

2. Set each data block based on the size of the search area 

3. Set the position of each data block, and moving step according to a 
distance from a control point 

4. Set the size of each data block according to a distance from the control 
point 

5. Set the size of each data block based on a correlation value of the control 
point 

1. Setting of search area- inclusion square 

Fig. 18 is a view (l) illustrating the setting of a search area. 

Basically, with regard to the search areas of left and right images, 
mutual orientation has been carried out in step S30. Thus, vertical 



parallax has substantially been eliminated. In the drawing, the respective 
points CLl and CRl of the left and right images are present on the epipolar 
line Pi; CL2 and CR2 on P2; and CL3 and CR3 on P3. As shown, if the 
points measured by the survey instrument form triangles of CLl, CL2 and 
CL3 in the left image, and of CRl, CR2 and CR3 in the right image, search 
areas are square areas including the respective triangles. Here, 
explanation is made of, for example, a case where the left image T in the 
area is set as a reference data block (template), and a position corresponding 
to this image is searched in the right image. In this case, if the right image 
including a search data block is set as a search area S and vertical parallax 
has been eliminated, searching is executed on the epipolar line P3. If the 
elimination of the vertical parallax has been incomplete, then searching is 
executed also around the corresponding epipolar line. Such processing is 
carried out on each line in the inclusion square area including each control 
point. When set as shown, an area overlapped with an adjacent triangle is 
formed. However, a triangular area of other than the control point can be 
surely searched without any mistakes. 

Now, Fig. 19 illustrates a search area. The area inside the triangle 
of Fig. 18 may be set as a search area. However, it is uncertain whether a 
corresponding point other than the control point is in the triangular area or 
not. Accordingly, as shown in Fig. 19, it is efficient to set an area including 
the triangular area as a search area. 

Fig. 20 is a view (2) illustrating the setting of a search area. 
As shown in Fig. 20, when searching is executed for the positions of 
search data blocks corresponding to respective reference data blocks 
(templates) Tl and T2, search areas Si and S2 may be provided with respect 
to each of the overlapped templates in the search area on the line P3. Here, 



as an example, areas within a specified range from the centers of the 
reference data blocks Tl and T2 are set as search areas Si and S2. 
Apparently therefore, the area nearer from each control point can be 
efficiently searched, and reliability can be enhanced by overlapping. A 
plurality of search areas may be set for each template. Further, search 
areas near CRl, CR2 and CR3, that are the corresponding points of the 
control points in the right image, may be set small, and gradually larger as 
they are separated farther away (a lowest or highest value may be set as 
occasion demands). In addition, the size of the search area with respect to 
the data block in the search area may be properly decided depending on the 
position, moving step or size of each data block described later. In this way. 
the range including the triangle can be efficiently searched. 

2. Setting of each data block based on size of search area. 

Figs. 21(a) and 21(b) are views, each thereof illustrating the setting 
of the size of a search area. 

As shown in Fig. 21(a), since the size of a triangle as a search area 
varies depending on the points of measurement, the size of a data block is 
set to match each triangular area. For example, each data block is decided 
on the basis of an inclusion square (see Fig. 18) set as a search area as 
described above. With respect to such a circumscribed square, the size of 
1/k is set as a data block. A coefficient k maybe fixed beforehand, or 
properly decided according to accuracy to be required and the size of an area. 
Another example is shown in Fig. 21(b). In another example, a radius r of 
the inscribed circle of a triangle is obtained, and r, or 1/2, 1/3 or the like of r 
may be set as a data block size. The radius r is obtained by the following 
expression : 
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r=V~{(s-o)(s-p)(s-q)/s} 
Here, s=l/2(o+p+q) 



(11) 



(12) 



The codes o, p and q respectively denote the lengths of three sides of the 
triangle, and these are calculated by measurement coordinates of the three 
5 points. 

The method of obtaining the data block size is not limited to these, 
and any can be employed as long as it is suitable for the size of each triangle. 

3. Setting of position and moving step of each data block according to 
10 distance from a control point 

Figs. 22(a) and 22(b) are views (3), each thereof illustrating the 
setting of a search area. 

3.1 Position of search data block with respect to reference data block 
15 As shown in Fig. 22(a), for example, if Tl on the left image is set as a 

template (reference data block) and the same image is to be searched from 
the right image, the position of Tl is searched within a predetermined range 
on the epipolar line p3 by setting a point SI as the center of a search area, 
the point SI being distributed at a distance of a and b from CRl and CR3 on 
20 the epilpolar hne P3, according to the distances a and b from the control 
points CLl and CL3. Accordingly, since searching is executed in the 
vicinity of the SI area estimated as the corresponding point of Tl, 
processing can be carried out efficiently and within a short time. 
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3.2 Position and moving step in search area 

For example, as shown in Fig. 22(b), searching is more efficiently 
carried out by making the moving step coarser as it is moved farther away 




toward both sides from the center of the search area set in 3.1. In other 
words, the moving step in a part near the center of the estimated area is 
made fine, the moving step in a part farther away is made coarse, thereby 
enabhng efficient searching to be carried out. For the ratio of making 
coarser, coefficients may simply be set, or the ratio may be changed in 
proportion to a distance firom the gravity center position of the triangle (see 
3.2.1) or to a ratio of nearer one of the two control points (see 3.2.2). 

3.2.1 Triangle center of gravity method 

Now, description will be made of a method of deciding the position 
and the moving step of each data block based on the gravity center position 
of a triangle. Figs. 23(a) and 23(b) are views, each thereof illustrating the 
center of gravity of a triangle and a data block. 

As shown in Fig. 23(a), distances from the respective control points 
to the gravity center position G of the triangle are equal to one another. 
However, a distance from a point T other then the center of gravity to a 
certain control point may be shorter than that firom the center of gravity to 
the control point. Thus, the position of the center of gravity is set as a 
maximum size of the data block, and a step is reduced as it is set closer to 
each control point. With Cl (xl, yl). C2 (x2, y2), C3 (x3. y3) and T (x, y), 
the distances of the respective points firom T are as follows: 
T-Ci:Ll=^{(x-xl)2+(y-yi)2} (13) 

T-C2:L2=^{(x-x2)^+(y-y2)2} (14) 
T-C3:L3=^{(x-x3}2+(y-y3)2} (15) 

The distances of the respective points fi-om the position G of the center of 
gravity are as follows: 

G-Cl=G-C2=G-C3=Lg=V~{(xg-xl)='+(yg-yl)2} (le) 
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A minimum one is obtained among the distances LI to L3 calculated by the 
above expressions (13) to (15), a ratio between this distance and a distance 
toward the center of gravity is obtained, and then a moving step is decided. 
For example, if the maximum and minimum step sizes of the gravity center 
position G are respectively set as SGT and SMIN, then a step size at the 
point of a smallest distance L (L2 in Fig. 23(a)) from the template position is 
obtained as follows: 

Step size=L/Lgx(SGT-SMIN)+SMIN (17) 

3.2.2 Distance method from ratio of two nearer control points 

Proportional distribution is made according to distances (of two 
nearer points) from the respective control points to T, and a step size is 
varied according to a distance to be searched. For example, a step size is 
decided based on the ratio between two smaller distances among the 
distances of the expressions (13) to (15). As shown in Fig. 23(a), with 
L2<L3<L1, the point of L2=L3 is set as SMAX, a maximum step, and the 
following is set: 

Step=L2/L3xSMAX ^1^^ 

The method for deciding the position and the moving step in the search area 

is not Umited to the above. 

4. Setting of sizes of each data block and search area according to the 
distance from a control point 

4.1 Distances from the respective control points to an image position to be 
set as a data block are obtained, and a data block size is varied depending 
on the ratio of these distances. As shown in Fig. 23(b). distances from the 
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respective control points to the gravity center position G of the triangle are 
equal to one another. However, a distance from a point T other than the 
center of gravity to a certain control point may be shorter than that from the 
center of gravity to the control point. Thus, the data block of the gravity 
center position is set as a maximum size, and an area is reduced as it is set 
closer toward each control point. With Cl (xl, yl), C2 (x2, y2), C3 (x3, y3) 
and T (x, y), the distances of the respective points from T are calculated by 
the foregoing expressions (13) to (15). The distances of the respective 
points from the position G of the center of gravity are also calculated by the 
foregoing expression (16). Then, a minimum one is picked up among the 
distances Ll to L3, the ratio between this distance and a distance toward 
the center of gravity is obtained, and then a data block size is decided. 

For example, if the data block size at the gravity center position G is 
GT, and the data block size at the point of a minimum distance L (L2 in Fig. 
23(b)) from the data block position is CT (data block of C2 in Fig. 23(b)), 
then the following is established- 
Data block size=L/Lgx(GT-CT)+CT (19) 

With regard to the vertical and horizontal widths of the data block, if 
vertical parallax has already been ehminated substantially by orientation 
processing, then the width in the vertical direction may be non-variable and 
fixed, while only the width in the horizontal direction may be varied. 
Accordingly, in the vicinity of the control point, a data block is set small 
assuming that a corresponding point is present nearby, and the presence of 
a corresponding point becomes uncertain as the position becomes farther 
away from the control point, thereby a data block at such a position can be 
enlarged to efficiently and surely obtain a corresponding point. 

The method of deciding the size of the data block is not Umited to the 
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above. However, the above method assures the efficient and fast execution 

of searching. 

5. Setting of data block size based on correlation value of a control point 
5 Figs. 24(a) and 24(b) are views, each thereof illustrating the size of 

each data block. 

The vertexes of each triangle (control points) are points that have 
been measured, and thus correspondence has already been established. 
Accordingly, correlation coefficients are obtained for various template 
10 (reference data block) sizes at these respective points (as shown in Fig. 
24(a)). and a template having a highest correlation coefficient is set as a 
template image. Based on the image of the obtained template size, 
searching is executed inside an area near each control point. The template 
size obtained at each control point is varied according to its position to be 
15 searched. 

As an example, proportional distribution is made according to the 
sizes obtained at the respective control points and the distances (of two 
nearer points) from the respective control points, and each size is varied 
according to a distance to be searched (see Fig. 24(b)). If CI (xl, yl). C2 (x2, 

20 y2), C3 (x3, y3), and a template T (x, y) are set. template sizes are obtained 
by the foregoing expressions (13) to (15). Accordingly, each template size is 
decided based on the ratio of the two smaller distances that have been 
obtained. For example, as shown in Fig. 24(b), with L2<L3<L1, the 
template sizes of CL2 and CL3 are divided by the ratio of L2:L3. and set as 

25 T sizes. 

With regard to the vertical and horizontal widths of the data block, if 
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ve.Uca. p«aUax has subs— been el— d by onentation 
passing, the width in the vertical direction nray be non variable and f.ed, 
while only the width in the horizontal direction may be varred. 

Ac^rding to the present invention, the execution of unage 
correlation processing based on a triangle exposed of control points 
eliminates the necessity of preparing a number of images drfferent from one 
another in resolution. As a result, it is possible to carry out high-speed and 
highly reliable image correlation processing. Moreover, srnce measurement 
can be carried out also by using the survey instrument, the measurement 

measurement has conventionaUy been impossible because it takes a long 
time, fast and easy measurement can be carried out. 



